-
Notifications
You must be signed in to change notification settings - Fork 102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
providers: support for proxmoxve #1023
Conversation
Thanks for the patch! For reference, Proxmox support has also been discussed in Fedora CoreOS in coreos/fedora-coreos-tracker#736. And while it's not currently being actively pursued, I don't think that should prevent support for it to land in Afterburn. That said, I've tagged the ticket to be discussed again in the next community meeting, at the very least so that we can agree on the platform ID. Feel free to join if you're able! Are you planning to also implement the Ignition side of this? |
Hi @jlebon, Cheers. |
for the record we are implementing the ignition support of the |
This makes use of coreos/afterburn#1023 to set up any static networking from the initrd (for Ignition) and the hostname (early enough so that Ignition could overwrite it).
This makes use of coreos/afterburn#1023 to set up the ssh keys from the metadata.
This uses coreos/afterburn#1023 to provide Proxmox images. TODO: Add Ignition PR.
This uses coreos/afterburn#1023 to provide Proxmox images. TODO: Add Ignition PR. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup.
This uses coreos/afterburn#1023 to provide Proxmox images. TODO: Add Ignition PR. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup.
This uses coreos/afterburn#1023 to provide Proxmox images. TODO: Add Ignition PR. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup.
From the Flatcar side we would also be interested to have this. One more note about network units: This seems to be netplan - do you want to focus on a subset or support the full netplan spec? Recently I added a |
We should find a way to support ignition in |
According to the discussions in coreos/fedora-coreos-tracker#1652 and coreos/fedora-coreos-tracker#736, it's possible to provide an Ignition config but currently quite awkward to do. I think this path is probably the best compromise, though maybe let's discuss in coreos/fedora-coreos-tracker#1652 since that's where all the context currently is? |
Still need to review this, but can we add a new docs page (e.g. under https://coreos.github.io/afterburn/usage/) to describe the situation on this platform a bit more? Basically:
|
FYI: I don't think we can use this in Flatcar when it relies on cloud-init userdata because Flatcar will run coreos-cloudinit anyway if it doesn't see Ignition user-data and then we would have it processed twice. For the network config that can be quite confusing because the afterburn network unit would only support a subset. |
I will let @arcln answer for coreos-cloudinit and focus my answer on the network side of things. Proxmoxve has a pretty limited Software Defined Networking (since the v7.4 as far as I can remember). To my knowledge, this is not widely used. This will probably improve in the future.
For this second option, on the ignition side of things ... my understanding is that network should be (must be ?) configured prior to ignition being launched. I remember having a conversation on systemd-sysext extensions for flatcar where an http url was used. Ignition was processing it so early that it was done before static network configuration. Systemd-sysext where therefore not installable from a network repo through Ignition. A better option may be: It is not clear to me if Flatcar fully endorse Afterburn, could you confirm ? |
The static IP addr setup through the kernel args inside Ignition is only needed if Ignition should download something. Using afterburn for that would only be possible if there was support for proper metadata at Proxmox VE. With the proposed approach it wouldn't work to use afterburn because the userdata is Ignition JSON then ;) If you have a chance of getting into discussions with Proxmox to define metadata separate of userdata, that would be nice. Even if it would be cloud-config-based metadata as done for VMware, because there one can use the netplan CLI option in afterburn to extract this (A supported afterburn subset might also be ok as alternative) and one could then get initrd networking that way while using Ignition userdata. |
I do not specifically agree, let me precise my throughts : Also some users would override userdata with ignition configuration, but some others won't. Again this PR should work ok in both cases. |
From my understanding, Also |
Ah, right, it has two separate files and already roughly follows https://cloudinit.readthedocs.io/en/latest/reference/datasources/vmware.html#walkthrough-of-guestinfo-keys-transport similar to the case with the special vmware guestinfo key. One could ask proxmox to set
Thanks for checking that, yes, the config drive mechanism will be triggered on Flatcar if not explicitly turned off. We could decide to stick to your afterburn support instead of handling this in coreos-cloudinit and thus say that coreos-cloudinit won't be supported on this platform. What I think would be a compromise for Flatcar is to use afterburn for the Anyway, since afterburn is always opt-in for distros it won't hurt to merge the PR (after the afterburn/src/providers/vmware/amd64.rs Line 70 in f918f4c
|
We agree that netplan support is important for Afterburn to be working on various OS. I will work on that in the next few days. |
This uses coreos/afterburn#1023 and coreos/ignition#1790 to provide Proxmox images. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup. The afterburn support for the SSH key and hostname parses the user-data when it's cloud-init. The coreos-cloudinit support is not there but can be added in addition: We need to add a new provider that varies from the existing config drive support because the file is called user-data and not user_data, and it needs to look for a filesystem label cidata and not config-2.
This makes use of coreos/afterburn#1023 to set up the ssh keys from the metadata.
This makes use of coreos/afterburn#1023 to set up any static networking from the initrd (for Ignition) and the hostname (early enough so that Ignition could overwrite it).
This uses coreos/afterburn#1023 and coreos/ignition#1790 to provide Proxmox images. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup. The afterburn support for the SSH key and hostname parses the user-data when it's cloud-init. The coreos-cloudinit support is not there but can be added in addition: We need to add a new provider that varies from the existing config drive support because the file is called user-data and not user_data, and it needs to look for a filesystem label cidata and not config-2.
This uses coreos/afterburn#1023 and coreos/ignition#1790 to provide Proxmox images. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup. The afterburn support for the SSH key and hostname parses the user-data when it's cloud-init. The coreos-cloudinit support is not there but can be added in addition: We need to add a new provider that varies from the existing config drive support because the file is called user-data and not user_data, and it needs to look for a filesystem label cidata and not config-2.
This uses coreos/afterburn#1023 and coreos/ignition#1790 to provide Proxmox images. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup. The afterburn support for the SSH key and hostname parses the user-data when it's cloud-init. The coreos-cloudinit support is not there but can be added in addition: We need to add a new provider that varies from the existing config drive support because the file is called user-data and not user_data, and it needs to look for a filesystem label cidata and not config-2.
This uses coreos/afterburn#1023 and coreos/ignition#1790 to provide Proxmox images. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup. The afterburn support for the SSH key and hostname parses the user-data when it's cloud-init. The coreos-cloudinit support is not there but can be added in addition: We need to add a new provider that varies from the existing config drive support because the file is called user-data and not user_data, and it needs to look for a filesystem label cidata and not config-2.
This uses coreos/afterburn#1023 and coreos/ignition#1790 to provide Proxmox images. This pulls in flatcar/bootengine#91 and flatcar/init#115 to run afterburn for hostname, network, SSH key, and metadata attribute setup. The afterburn support for the SSH key and hostname parses the user-data when it's cloud-init. The coreos-cloudinit support is not there but can be added in addition: We need to add a new provider that varies from the existing config drive support because the file is called user-data and not user_data, and it needs to look for a filesystem label cidata and not config-2.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments, but this looks sane overall.
Everything looks good to me, I think this can be merged. @pothos asked for netplan support. I implemented it, should I push to this PR ? |
Can you squash it down to one commit and also address the lints from CI?
Let's do that in a follow-up PR? We'll want to add docs as well for this (see the similar VMware-related netplan support). |
Fixed clippy errors and squashed commits. I'll open a netplan support PR after this one is merged. |
Hmm, Jenkins is having trouble merging the release notes for testing:
Can you rebase onto the latest main? |
(I tried rebasing it myself, but it looks like that option to allow maintainers to push was disabled.) |
allow reading cloud-init drive from proxmoxve. supports writing hostname, sshkeys and networkd configuration.
Head branch was pushed to by a user without write access
done |
Cool ! We will work on the netplan PR + focus on the ignition and flatcar side of things :) |
This makes use of coreos/afterburn#1023 to set up the ssh keys from the metadata. Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
This makes use of coreos/afterburn#1023 to set up any static networking from the initrd (for Ignition) and the hostname (early enough so that Ignition could overwrite it).
Hello,
This PR adds support for reading the cloud init drive from a ProxmoxVE instance.
The code mounts
/dev/disk/by-label/cidata
to a temprary location and then parses the YAML files present inside.It supports : setting hostname, writing ssh keys, and writing systemd network configuration.
The changes introduces a new OEM ID
proxmoxve
that will be matched from the kernel command line. As time of writing, this OEM ID is not yet recognized by Ignition or other tools.We currently maintain a build script that modifies a Flatcar image to inject this version of Afterburn : https://github.com/enix/flatcar-proxmoxve
Feedback is welcome :)